> 



docoheIjt restJhj 



SS 118 398 

AOTHOB \ 
TITLE 

FOB DATE 
NOTE 



r 



u 



SB 020 212 



EDES PEICE 
DESCEIPTOBS 



IDENTIFIERS 



^ABSTEACT 



Goss, L. D..; Cxoft, F. M.' 

E\raluati,on of Inn'ovative Basic Graphics • ' • 

iTistruotion. . 
Jun- 75 . i . • , . 

30p.; Paper pre sern ted at the Ahnual Meeting o-f the ■[ 
American Society for Engineering Education (Colorado' 

a -State .University, Ft. Collins, Colorado, June 16-19,. 

; 1975) - , ' ■ . 

MF-$0.83 HC-$2^06 Plus Postage . 

♦Achievement; *Educatiohal Besearch; Educational 
Television; *Engineering Education; Evaluation; 
♦Giapiiic Arts; Higher , Education; Individualized ^ ( 
Instruction J Instruction; lecture; *Teaching 

Hethod'^s s ' , ' 

• Besea^fch'' Reports ^ . 



'This paper describes the results of a three-way 
evaluation of 'traditional, telev,ised, atfd individually- paced 
instruction •in beginning graphics/courses • The purpose of the study 
was to^det^rmiftQ mean gain scores; for students in each of the three 
groups^ In oi^der to- disguise the experiment, no attempt' waa made to 
randomly assign subjects to groups, nor were subjects told they were 
participating in an experiment. Two forms of a standardized tesll wej^ 
used; in a pretest-posttest format. The mean, standard deviation, 
scatter diagram, and .regression line were calcul3.ted foir^^aph 
instructional group and appear in attached appendices* Histograms are 
used to show pretest and J>osttest scores for subjectjs in each of the 
groups and s:Qitter diagrapis are constructed in which line^ of 
regression' a£e drawn* With the use 'of the stardard e;rrors/ of 
measurement for each of the instructional groups, it was/found that 
significant achievement resulted for 73 percent of the traditional . 
giroup,,78 percent of the.' televised group, and 59 percent of those 
subjected to individual instruction, *Some possible explanations for 
these phenomena are provided • (AutKor/CP) " / 



. X- . . . 

* ' jDocumenVs acquired by ERIC include many informal unpublished * 

* materials not available from other sources • ERIC makes eVery effort * 

* to obtain the best copy available,- Nevertheless, itemsr of ftargin^tl * 

* reprodu.cibility ate Soften encountered and this affects the quality * 

* of the microf iche^and hardcopy te productions ERIC makes avallabile' * 

* via thfe ESIC Document Reproduction S.ervice (EDRS)\ EDRS is not ^ * 
^ responsible for the quality of the original document* Reproductions * 

* supplied by EDRS are the best that can be made from the original, - * 



U.S. OEPARTMENT OF HEALTH, 
EDUCATION A WELFARE 
NATIONAL INSTITUTE OF 
EDUCATION ^ 

THIS OOCUMENT HAS BEEN REPRO 
OUCEO EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION ORIGIN. 
ATING IT POINTS OF VIEW OR OPINIONS 
STATEO DO NOT NECESSARILY REPRE 
SENT OFFICIAL NATIONAL I ^STLLP ^E OF 
EDUCATION POSITION OR POCTCY 



EventNurriber 1640 

. : ^ ' 

, AMTO^CAN SOCIEIY FOR EMGINEERING EDUCaTION 
ANNUAL CXXSIEERENCE, JUNE 16-19, 1975 

— — T ^ ^~ \ . 

CDLORflDO STATE' UNIVEPSITy ^ 

OOUilNS CO 80521 ^ • ^ 

TITLE^OF PAPER Evaliiation of Innovative Ba^c Graphics Instxuction 

NAMEOF AUJHOR L. D. Goss^d F. M. Croft 

TITLE AND ADDRESS OF^ AUTOOR Associate Professor 

Division pf Technology 

Indiana State University Evans ville 

Evansville, Indiana 

Instructor 

' Departnent of Graphics and Design 

. ^ ■ West Virginia Institute of Technology 

^ Montgone^y, West Virginia 



graphics cxDurses, Hie three methods of instruction already: existed 



s ' . - . , . - • 1 

-^Hiis paper describes ^the results of a three-way evaluation of tra- 
ditional, televised, and individually-paced instruction in beginning. 

;at 

West Virginia Iiistitute of Technology and have been in use in the balsic 
gjsphlcs courses for*several years; thei^fore the methods, materials, 
and instructors were seasoned and "debugged" to the point that the' 
faculty of the Graphics and Design department treated both the courses 
and the methods of instruction as "off-the-shelf " programs which had 
•becomemore or less standardized through the years. part of our own 
concern about student^ progress and accomtability,; w^^indertook the ex- 
periinent which this paper docunents. We did not set out to "prove" any 
particuX^r me thod of ir^trudtion nor to rank the three methopLs we were 
using, but rather siitply attenpted to detennine whether -the teaching 
methods affected* student achievement. 

-jr. . 

Ttie traditioncil method of instruction met in a lecture-labora- 
tory setting. , Both lecture and laboratory /were con<^\acted in the same ' 
roan with no scheduled break between the two functions. The ^materials 
used for the lectures coilsisted p£ OGfitBSfciaW 

transparencies and . chalkboard drawings ^f or ©camples of standards, appli- 
cations, and procedures to be followed in executing assigned problems. 
Progress thipugh the course was "lock- step" in that all tests, quizzes,, 
and other evaluation procedures ^/ere conducted for the entire class as 
a unit. ^ ' ^ > 
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Hie^tefevised instruction varied /frcxti the traditional method only 
inptesentation form. Extreme- reliahob was placfed on live execution of 
drawing sainples on a drafting table J picked vp by an overhead camera and 
broadcast thx)ugh video monitors. mLs method of presentation was aug- 
mented by cormercially ppqpared vidfeo tapes. Otherwise, the method of 
instruction wis handled exactly the same as the .traditional way. . 



The instructional format of individually-paced instruction was a 
radical departure from the other two -methods. Its basic attributes are: 
\ ^;v ^^ll^ormal presentations are accxmplished thfoiagh recorded 
. • msdia - films,^ slides, tapes, etc. 

no attenpt is made to mate all students study the same to- 
rpids simultaiieously. . ^ * 
all testing and eyaliaation is. done on an individual basis. 
Ihere area number of other detaiLs which set this irethod apart f ran- the 
others, but these are the fundarrental differences. ATI thre^ methods 
had text' readings and assigned praptioe problefns as comon 'vehicles for 
instruction. ' . , • 



It was our purpose to disoo\fer hew these different teaching me- 



thods ^effected strdent achievement. We did not have a clear cut ^ 
perimsnt in that there were undoubtedly sore variables over vAdch we 
■\ could ix:>t exercise control sudi as the possibility .of certain types of 
\ students all .being -scheduled^ irito specific sections of the courses based 
Cn the avaiiatllity of differential subjects outside of the department; . 
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such as math, chendstry or English. We' did not want a contrived ex- 
perijTiental situation to exist in the Classes as they were being con- 
ducted, so we made no attgnipt to redistribute the students in different 
classes but siorply we took, thpn as^ thqy were; recognizing that sucSh 
differences in population could exist. 

K 

The logical procedure for chec3d.ng student afchieveiTBnt is through 
a pretest-posttest fonnat. Ihxs method yields information about the 
students' knowledge prior to taking the prescribed instruction and then 
gives accnparison with their performance on the same, or a similar test 
at the end of the instructional. period. We chose to us^ this procedure 
inthis e^qperiment. . / ' 



Since two forais of the standardized test which we chose exist, and 
we wanted to d^rmine the reliability coefficient between the two, 
each course section which was selected to be in the ej^riment was sub- 
divided into fourdifferignt groips.* This allcwed adrrdnistration jof the 
tests in A-B, B-A, A-A, and.B-B sequence ;using either the same or alter- 
nate formoi the test in a pretest-posttest format with ^ach student. 

TixLs was done as follows: on the secx)nd class meeting of the semester an 

' / ' ' ' ' 

eq\aal niinber of the two test forms were shuffled' and then distributed to 
the students in the order ih which they were sej^ted (which was by indi- 
vidual selection, of the students.) The answer sheets were collected in^ 
.the order ih which the students. finished the test and sorted by form fp|: 
grading. Each subset of answer sheets was then dealt into two classi- . 



• fications and coded detennihe which form of the test the individual 
sttjdent would subseqiiiently take as a posttest. • Uiis allowed us to t^ke 

, ^ ^ : ; . ^ 

any variations- in difficulty of the test form into account in the sta-, 
tistical ^alysis. *t 

* ! I 

t 

The mean, standkrd, deviation, scatter diagram, and regression line 

; / ! 

were, then calculated and plotted for each of seven distributions: A-B, 

B-A, A-A, and B-^B formats, television instruction, traditional in- ; 

struction, and indivi,diaally-paoed instruction* Of thesis , the first 

four were sinply used standardization purposes am the last three 

are the ccnparisons in which we were interested* Additional correla- 

tions 'oould have been targetted for analysis such.* as*: individual pro- <• 

fessdr's sjnjdents, distributions by academic major, by ethnic or eoo- 

Qajiic background, by prevjLous academic achievement, etc., tut we chose 

to limit the, investigation to a cotparison of the teaching methods only. 

Additional analysis of th^* three target distributions was also can^ 

pleted to determine^ whether any significant differences between pretest 

* 

and posttest for each situation ^ahd among pretests and pogttests for 
all three teaching methods wasted. 

r 

The instrument we chose to use to gather data for this e^^eriment 

is the Drawing sectio|^ of the Cooperative IrtdustriaJ. Arts Tests, pub- 

• » 

lished by Educational Testing Service. This test was originally de- 
signed as an achievement test 'for junior high school* students , but . 
since its normative data indicates that it is probably a little 



difficult for th:|^ groi:p and since most of the college bound students 
in Vfest Virginia receive no training in instrument drawing in the public 
schools, we felt justified in using it for this experiment. Not of 
least significance in the decision to use this particular test is the 
fact that it is. the only one in this subject area listed in the Mental 
Measurements Yearbook. We felt this fact was significant if for no 
other reason than ease of replication of the ej^eriment by others at 
their cwri institutions. As has been stated previously, the test exists 
in two forms. It is a timed, objective, multiple c^ioice' type exam which 
covers both the theory and application fundamental engireering draw- 
ing. Hie students had very little difficulty in answering the 50 ques-- 
tibns within the 35 minute time limit, so ,at the college level it could 
easily be classified as a "pcwer" test, ratdier than a "speed" test. 
Its subject matter is fairly ccnprehensive, but the test does have two 
distinct drawbacks; 1) its standardization population i^s not generali- 
zable to college age stix3ents and 2) there is a lew- reliability correT 
lation (.70) between the two forms of the test. Sufficient information 
is included with the test* instructions for individuals to establislj 
normative data for the students at their own institution, but the lav 
correlation between forms *must be rectified by other means. As men- 
tioned pre;^ously, we chose to conbat this jJroblem by organizing the 
population into subgro\:ps and utilizing all four permutations for the 
order. of administering the forms. TIlLs procedure tends to level out 
any differences between the tests. ' "^''^ 



All o£ the 



»?t35ents, who participated in the experijnent maj6r- 



ing in. one of th6 ' following academic area§; ciy?.l engineerings, ele 
trical engineering, cjieannical^^m sciences ^ meet 
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engineering, industrial arts education, civil engineering -feeohnoL 

f 

drafting and design engineering technology , electrical engineeri 

* * * ' 

.technology, mechanical eiigineering technology, or mining engineeri 

technology. With rare exception, they were all first semester fi^esiimen 
who had graduated from high school two and one-half months prxorj to the 
beginning of the experiment. l!he pretest was administered to the en- 
tire enrollment in the fundamental drafting courses, but since ^ataaral 
attrition prevented many of the stvidents fran finishing the comrse, ^the 
total population for the e3^>erjjnent was not known until after tie post- 
test had been^ given. 203 students ocnpleted the posttest. The;^ were 

< 

divided info groips of 72, 65, and 66, for traditional, tele^fision,*^ and 
individually-paced instruction respectively. 



The e:q>eriinent enocnpassed two multi-section courses in furldamental 
^i^ineering drawing which had similar sets of objectives. Both courses 
are concerned with establishing the theory and practice of instrrument 
drawing fran the fundamentals of orthographic projection throxagh the 
execution of catpleted detail drawings. Uie main difference in the 
couri3es is that' of contact hours devoted to laboratory time. Enrollment 
according to designated major is different for the courses- based vpon 
the cojitact hours, with engineering majors being preidaninant in the four 
contact hour course, and engineering technology and industrial arts 
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majors in jthe six contact hour course. Neither tof the courses is 
taught exclusively by any single instructor f rather, the course load is 
spread as etvenly as possible among three different instructors. I^ke- 
wise no ins tructor used any of the three teaching metho^ excliilively , 
- but each ore used the methods which suited the enrpllinent and class- 
roan oo^idiiion. *By chance nearly equal populations resiiLted for the 
three methcdologies. 

' ■ ■ ^ ■ .: ^ 

* / 

Tables 1, 2, and 3, (see Appendix) shbw the histograms of both 
the pretest and posttest results for the traditional, television, and 
individually-paced instruction, respectively.. Included in these tables 
are the cumulative totals and percentile rankings for the various test 
scores. .Not shown on these tables, but also of importance statistLc- 
ally, are the percentile ranges which can be assigned to each score 
based on the standard error of measurement of the test. According to 
Educational Testing Service, the standard error of measurenent of both 
forms of the test is a constant 3 raw score points. The percentile 
range which can be assigned to each score is determined by using the 
percentile associated with the raw scores which are three above and 
three below the score in question. For example the percentile rang^ 
for the raw score of 17 in -the pretest under traditional instruction is 
vfron the percentile for a raw score of 14 to the p^centile for a raw 
$oore of 20 or frqm 6 to 25. The standard error of measurement also _ • 
must be used in determining significance as will be discussed later. 



Figures 1, 2, arid. 3 show graphical coiparisons of the^retest and 
posttest distributixiDns for the traditional, televised, and* individu- 
ally-paced instruction methods, respectively. One thing which is 
easily discemable from either the histograms or the distributions of 
the data 'is that regardless of the method of instruction the students 
appear to have learned a considerable Amount as evidenced, by the shift 
of the distributions and the differences of means. 



Figures 4 and 5 shew catposite graphs ccnparing all three pre- 
tests and all three posttests, respectively. A test of significance 
on the variation of the means indicates in the pretest there is no 
significant difference' in the populations. However, in th^ posttest, 
traditional instruction was found to be significantly better, individu 
ally-paced instruction was significantly worse and television instruc- 
tion remained^ unchanged. Analysis of the variance of the standard de- 
viations indicate that both the traditional and individually-paced in- 
struction grovps are also significantly more uniform in posttest re- 
sults than is the televisicn instruction qroxsp. 

Of greatest significance in analysing the data we cxirpiled is a 
scatter diagram. Figures 6, 7, and 8 and Tables 4, 5, and 6 shew the 
scatter diagrams and associated data for traditional, televised and ' 
individually-paced i3istruction respectively. 

'\ Each of the scatter diagrams is a plot of paired scores for the 



pretest (abscissa) aaid po.stt3st (ordinate.) Results- of the entire 
popxalgition'^n each of the three groi:ps has been plotted. The solid 
diagonal line running fron the origin tcward the tfper right hand cor- 
ner represents pie status quo condition — the student scored identi- 
cally on toth tests. The standard error of measvirenvsnt lets us es-' 
tablish an envelope or range of scores which could be considered sta-* 
tus quo due to the possible variation of three raw score points vMch 
could Jbe e^^cted by any student taking an' inmediate retest in either 
the pretest or posttest situations. Thi^ envelope or range is deno- 
ted by the dashed lines ^above and belcw the status quo line. 

Fran this plot it is relatively easy to determine what percentage 

of the students scored significantly higher or lower on the posttest 

ocitpared. with the pretest. All three bopxiLations • had ^pproxijnately 

1.5% who scored significantly lower on^. the posttest. Significantly 

higher result^ were 73% for traditi(xial, 78% for televised, and 59% for 

individually-paced instruction. The number of students whose scores 

appear in the upper end of ' the status quo envelope are to be expected. 

They are the individuals who have had extensive traixijLng in the sxabject 

curea prior to enrolling in the coiarse. The individ^l^^in the lower 

end of the envelope are not as easily categorized, ffihlse may be'indi- 

viduals who lack motivation, who learn to respond to specifics rather 

than generalizations, who hav^ language and reading skills prc±)lems, 

or otherwise manifest problems as students. An effort must be made to 

•* 

identify these students and to deal with their problems if yc^ wish to 
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retaiji them in yqtoc program. - 
\ » . 



Mso' plotted on the scatter diagram , are the values of the mean 
>fpr the pretest and posttest and the calculated regression line which 
coxild be used for* predicting res tilts on t±ie posttest when only the 
pretest score is known, but which we are using as a cchparison indi- 
cator between poplalations . Ihe data in each accaipanying table give?' 
all J±ye nimerical infonnation necessary to plot the scatter diagram 
itself. • - * . 

Ccn^>arison o£ the. resxiTts yields sane very interesting ooncj.u- 
sions. As has alreacty been stated, the distributi9ns appear to have 
no significant differences between them ^rior.to instcubtion. All 
three groips of stuc3ents fared significantly better on the ^ posttest, ^ 
but the traditional ihstruction^metKod appeared to do better on th^ ^ . 
average than either of the other gcoips, and the individually-paced 

groip farsi-^significantly woirse. And, an analysis of -the variance of 

" * ^ ' . - , 

the standard deviations iixlicates that both the traditional . and indi- 
vidually-paced instruction groi;ps are also significantly more uniform 
in posttest results th^ is. the television ins tKlicf ion group. : 

o ♦ 

V.'' 

Does this mean that the tracittional \nethDd of instructton is m- 
herently better than individiaally-paced instaruction? It does not. 
Examination of the two scatter diagrams indicates .that different sub- 
groi^)S within 'each of the two populations ended xxp being in the status 
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quo envelope. For the traciitional instruction meiJiod, 'it was a groi:p 

•I ■ . . » _ •* * 

which scoriBd lower on the pretest while for the indiviSuaHy-paced 

/instruction grovp it consisted of students' who scored higher on taie, 

pretest. Students who knew little about the subject fared well under 

^individually-paced instructicn'but did not do as well under traLdition- 

^r-insttuGticm. "This is prcA)ably the result of Jiaving to pape the^ 

traditional course at a faster rats than these individuals can work ' ' 

to assiii]ilate\th§ knowledge. On- the other hand, students with prior 

kndwJLedge pf the subject apparently 'teixl to extend their kriowledge 

under /traditional instruction, but loaf mder individualIy-pad§S in- 

struction.. Thid w6bld indicate that performance cm the* p:^test could 

: " - ■ . - • . ■ ■ " ' ' \ - 

be used to differentially 'place students :ln a course of fered by . either 

. y ■ ' ■ ^ ' . ' - ]' 

,of the two* methods. The testing of such an hypothesis will hopefully 

be ,the subject of an additional paper on this topic to jDe p\±)lished in 

th^ near future. • 

What aboift television dLnstruction? %e fact that there i^ a 
wider spread to the data, as shown' both by the value of the standard. , 

■ ' ••- . ■ . ^ " : ^ ' " ■/ ■ . 

deviation ^and the slq^ of the regi:ession line, indicates that tele-r 
vision should not be an excliasive or habitual teaching medium in this ' 
Vubjecft. The criticism that has been levelled at "S^ame Street" and " 
:/ "The Elee-^iS Ccnpany" appears to be cppHcaMe/here as well. "The 
rich get richer, and the poor can't keep Television is a tran- 

sient nediura. It does not lend itself easily, as used in this experi- 
ment ,r to. review and sequence reinforcanent. Vfe are certain that there 
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must be Qtiier data avai^^le Which s\:5pDrtS the extensive vise of tele7 
vision. If SO/ we would welcome seeing the resvilts of ej^seramenta- 
. tiojfi, using the same ^tandarciized exam, v^ch would prove us v/rong. 
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TABLE 1 

' HISTOGRAM § PERCENTILES OF TRAUITIONAL INSTRUCTION 
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HISTOGRAM S f^ERCENTILOS. OF TELEVISION INSTRUCTION - , 
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